Accurate Shared Bottleneck Detection and File Transfer Delay Prediction for Grid Scheduling

Fakultät für Mathematik, Informatik und Physik Institute für Informatik Doctor of Philosophy by Muhammad Murtaza Yousaf Grid Computing promises a supercomputing-like performance at low cost, which means that a large number of users should have the opportunity to solve their problems on a widely accessible platform. In order to meet the claimed performance, the Grid research community needs to design methodologies which can exploit this computation power. Among others, one major issue in fully utilizing the Grid capabilities is assignment (“scheduling”) of appropriate resources for a job in a way that minimizes the execution time of a particular set of tasks. For a good allocation, which is crucial for efficient execution and use of Grid resources, a Grid scheduler must have comprehensive and up-to-date knowledge of available resources as well as the time it takes to transfer a certain amount of data between them. In this thesis, this need will be addressed with a system for predicting file transfer delay. This will be done by estimating stable network path characteristics without relying on the occurrence of other network traffic because of the unreliable nature of such measurements. The awareness of the complete network topology and particularly knowledge of congested network links which are shared by multiple flows can be of great use to Grid schedulers as well as many other applications. Finding an accurate solution for the detection of shared bottlenecks is therefore a main goal of this thesis. None of the state of the art Grid schedulers make appropriate use of such information about the network; however, more intelligent and efficient decisions can be made on this basis. In this thesis, it will be shown how the performance of a Grid scheduler can be improved by incorporating knowledge about file transfer delays based on shared bottleneck detection.